Pooka SDK Developer Guide
Application Presentation Model

Fighting Device Fragmentation - Working on a Virtual Device

Developing application for multiple platforms may be challenging because of the device fragmentation - the phenomenon that each single device differs in many parts, like processor architecture, operating system, screen resolution and orientation, sensors, peripheral and so forth. The most feasible method to fight with device fragmentation is virtualization, which abstract hardware components into abstract component.
In the above diagram, the left side shows that Pooka SDK® developers developers an application on a virtualized device and the right side shows that after the application is deployed on the device, it runs on the real environment. Pooka SDK® handles the mapping from virtual device to real device and ensure the application behaves consistently as much as possible across multiple platforms.

Device Abstraction

Device abstraction is the mechanism to abstract all real hardware and OS components into a virtualized component. A cross-platform application engine should usually abstract the following hardware for example:

  • Output Device
    • Display: physical screen
    • Speaker: speaker
    • (Mobile) Vibrator: vibrator
  • Input device
    • text input device: physical keyboard and soft keyboard
    • pointing device: mouse, track pad, track ball, touch screen
    • gaming pad: game controller and joystick
  • Capture device1
    • Image / Video capture device: camera
    • Sound capture device: microphone
  • Sensor
    • Geolocation sensor: GPS sensor
    • Motion sensor: accelerator
    • Attitude sensor1: gyroscope
    • Environment sensor1:ambient light sensor, pressure sensor, temperature sensor
  • Storage
    • Application sandboxed storage: application local storage, preference settings
    • Shared OS storage: account-based shared storage include image/pictures, camera rolls, sounds/audio, documents, and so on

 Note 
1 - These features will come in the next major version of Pooka SDK®

Presentation Abstraction

Among all types of device fragmentations, presentation fragmentation (display size, orientation, dot-per-inch, etc.) is the most challenging fragmentation that a cross-platform application needs to heal with. Most of cross-platform applications with visual presentation (except server-side programs or services) needs to present its content on multiple devices with consistency, Therefore the presentation abstraction is the most important virtualization in Pooka SDK®.

Presentation abstraction in Pooka SDK® is implemented with three components:

  • RealDevice
  • Virtual Device
  • Content

The following diagram shows their relationship and how a virtual presentation is finally mapped to the real device.

When Pooka SDK® application is launched, it does the essential initialization before entering application's own logic. The following flow diagram shows the initialization related to presentation abstraction. For simplicity, all other irrelevant initialization has been removed from the diagram.

App. Start
Initialize RealDevice
Collecte real device information, such as physical resolution, color depth, DPI, scaling factor
Initialize Virtual Device
Choosing the best virtual screen by best the matching aspect ratio
Initialize Content
Determines the content size, origin, mapping scaling, whether to apply "letter-box" padding strips
Start App. Logic

Real Device

The SDK RealDevice class represents a real device.
The RealDevice class is used to reveal the information on the actual device where the application runs. All the information revealed by this class are "real". For example:

  • physicalDisplayWidth: the physical display width
  • physicalDisplayHeight: the physical display height
  • screenRotation: the screen rotation
  • screenDpiX: The horizontal DPI (dot-par-inch) of the screen
  • screenDpiY: The vertical DPI (dot-par-inch) of the screen
  • supportMultiTouch: Whether the device supports multi touch
  • ...

Virtual Device

The SDK VirtualDevice class represents a virtual device.
The VirtualDevice class defines a virtualized device that developers "work on" to efficiently deal with the visual fragmentation cause by real device. In spite of huge amount of different screen size of devices on the consumer market, VirtaulDevice class defines only few basic types of "virtual screen" with specific base screen size and aspect ratios:

Aspect Ratio ⇒
⇓ Base screen
Square (1:1) Legacy TV (4:3) Legacy Phone (15:9) HDTV (16:9) Univisium (18:9) Ultra-wide (21:9)
QVGA
(Default Value)
240x240 240x320 (P)
320x240 (L)
240x400 (P)
400x240 (L)
240x427 (P)
427x240 (L)
240x480 (P)
480x240 (L)
240x560 (P)
560x240 (L)
VGA (x2) 480x480 480x640 (P)
640x480 (L)
480x800 (P)
800x480 (L)
480x853 (P)
853x480 (L)
480x960 (P)
960x480 (L)
480x1120 (P)
1120x480 (L)
SXGA (x4) 960x960 960x1280 (P)
1280x960 (L)
960x1600 (P)
1600x960 (L)
960x1707 (P)
1707x960 (L)
960x1920 (P)
1920x960 (L)
960x1706 (P)
1706x960 (L)
QSXGA (x8) 1920x1920 1920x2560 (P)
2560x1920 (L)
1920x3200 (P)
3200x1920 (L)
1920x3413 (P)
3413x1920 (L)
1920x3840 (P)
3840x1920 (L)
1920x4480 (P)
4480x1920 (L)

 Note  In above chart:

  • Aspect Ratio (the best matching one) is chosen by Pooka SDK® when VirtualDevice is initialized during application start-up
  • Base Screen can be chosen by developers inside application constructor. See VirtualDevice::baseScreen
  • It is highly recommended to use the default value VirtualDevice::BASE_SCREEN_QVGA for VirtualDevice::baseScreen . All sample projects are based on the baseScreen set to BASE_SCREEN_QVGA.
  • In every Pooka SDK® application, there are only several (6 in the current version) virtual screen layout that developers need to deal with.

Mapping Application Content from Virtual Device to Real Device

The SDK Content class defines the content space of a Pooka SDK® application.

During the application start-up, the Content class checks the information from real device and all available screen sizes from virtual device, then it tries to get the best matching virtual device, shown in the following figure. For example, the RealDevice is a smart phone with portrait screen layout, aspect ratio at (or close to) 16:9, then the virtual device with screen layout QVGA / 16:9 / Portrait is chosen. The virtual device has display size at 240x320 point (note: not pixel, see the next topic).

Since a virtual device that best marches the real device has been chosen, the application design work is done on this virtual device. Finally, when the application is deployed on the real device, the application content is scaled on the real device's display.

 Note  The recommended VirtualDevice::baseScreen value is VirtualDevice::BASE_SCREEN_QVGA , therefore the best virtual device chosen has a QVGA or wide-screen QVGA screen size. This screen size is the de facto smallest screen size on consumer market nowadays. As the result, the virtual-device-to-real-device mapping is actually up-scaled mapping.

Perfect Mapping and Imperfect Mapping

Although the virtual device best matches the real device is chose, its aspect ratio may or may not be same as the real device. There are two cases when doing the final content casting:

  • Perfect Mapping: the virtual device's aspect ratio is identical as the real device's, the content is simply scaled and cast to the real device.
  • Imperfect Mapping: the virtual device's aspect ratio is different from the real device's, the content is scaled and cast to the real device with padding strips applied at both sides or top and bottom, which is so called letter-box mechanism.

In current version Pooka SDK®, only six types of aspect ratios are available for virtual device, if the real device's aspect ratio falls in the middle between any of these available aspect ratios, an imperfect virtual-to-real mapping will happen and letter-box strips are applied. If these "black padding strips" are not acceptable for a particular device, then developers can set a custom wide-screen aspect ratio, which makes up a custom virtual device that ganerate perfect virtual-to-real mapping. See function VirtualDevice::setCustomWideScreenAspectRatio() for more details.

Always Design Application Content in Virtual Space

The basic rules for designing application content with Pooka SDK® is design the content in virtual space.

Points in virtual space

Points are smallest unit in virtual device. One point can be mapped to numbers of physical pixels in real device, which is called point size. The point size is the same as the Content 's scaling and can be retrieved by function Content::getScaling() or Content::getScalingFloat() .

Following are some examples of point size:

Real Device Size ⇒
⇓ Point Size
400 x 240 640 x 480 960 x 600 1366 x 768 1440 x 900
Point Size 1. 2. 2.4 3.2 3.6
Perfect Mapping? true true false true false

Pixels in real space

Pixels are smallest unit in real device, it is same as the physical pixel on real device's display.

Always Design Content by Points

To generate consistent application content on multiple devices, it is very important to remember to design the content by points in virtual device, not the by pixels. The following code snippet shows correct and wrong code when set a Sprite 's size and position, or draw a geometrical shapes:

Correct code:

// Set a sprite's position to (50, 20) and its size to (150, 80)
Sprite * pSprite = ...; //Get the Sprite
pSprite->setPositionPT(50, 20); //Set the position by points
pSprite->setSizePT(150, 40); //Set the size by points
...
// Draw some geometrical shapes:
GraphicsEngine * pEngine = GraphicsEngine::instance; //Got the engine instance
pEngine->fillEllipsePT(135, 200, 90, 75, Color::BLUE); //Fill a blue ellipse by points
pEngine->drawRectPT(125, 70, 80, 60, Color::RED); //Draw an red rectangle by points
...

Wrong code:

// Set a sprite's position to (50, 20) and its size to (150, 80)
Sprite * pSprite = ...; //Get the Sprite
pSprite->setPosition(50, 20); //Set the position by pixels
pSprite->setSize(150, 40); //Set the size by pixels
...
// Draw some geometrical shapes:
GraphicsEngine * pEngine = GraphicsEngine::instance; //Got the engine instance
pEngine->fillEllipse(135, 200, 90, 75, Color::BLUE); //Fill a blue ellipse by pixels
pEngine->drawRect(125, 70, 80, 60, Color::RED); //Draw an red rectangle by pixels
...

Conversion between Points and Pixel

There are functions provided to convert between point and pixel

The following code demonstrates drawing a bitmap-banner with length of the application content's width, height of 40 points, aligned to the top and horizontally centered in the content view:

int bannerL = Content::width - Content::pointToPixel(20); //Get the banner length, in pixels
int bannerH = Content::pointToPixel(40); //Get the banner height, in pixels
Bitmap * bmpBanner = ...; //Get the banner bitmap
GraphicsEngine * pEngine = GraphicsEngine::instance; //Get the engine instance
pEngine->drawBitmap(bmpBanner, (contentW - bannerL)/2, Content::pointToPixel(5), bannerL, bannerH); //Draw the banner
...

Cross-platform Content Design Strategey

Pooka SDK® helps developers abstract the real devices with hundreds of different display size and aspect ratio into few types of virtual devices. Each virtual deice represents one type of display with certain aspect ratio and orientation. At the develop time, developers needs to determine how to design the application content for these virtual devices.

For example, one design approach could be design all content layout dedicated to each screen orientation and each aspect ratio, therefore there are about 11 content layouts need to be designed.

Another simplified approach could be that design only three content layouts for square, landscape and portrait screen orientation.

An extremely simplified design approach is applied in most of Pooka SDK® sample projects for simplicity purpose, which is just design a single content layout used as a universal layout for all display types. In this universal design, only the "intersection part" of all display with different orientations and aspect ratio is in use to host content, which can guarantee the application content are fully visible on any types of display. The biggest disadvantage of this universal design is that there is large empty portion on some display left unused. This design with extreme simplification is not recommended for commercial application development.

For further reading about this topic, see Content Layout for Multiple Display.